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(54) Abstract Title 

Control of display by movement of handheld device 

(57) A handheld display device in the form of a PDA 15 is provided, the PDA comprising a liquid-crystal 
display (LCD) screen 3, a processor 5, a local memory 9, an input port 1 1 and first and second sensors 17, 19. 
Image data received from a remote source is transferred to the processor 5 by means of the data port 1 1 The 
processor 5 outputs the image data to the screen 3, with the updating of the image being controlled by signals 
generated by the first and second sensors 17, 19. Movement of the PDA 15 in a substantially vertical plane is 
detected by sensor 17, which causes the processor to update the image in a zooming effect, depending on the 
direction of movement. A tilting movement of the PDA 15 causes the processor to update the image in a 
scrolling effect, the direction of scrolling corresponding to the direction of tilt. 
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At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy 
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1 

A Handheld Display Device 

This invention relates to a handheld display device, and particularly -to a handheld 
display device for displaying images on a screen. 

Handheld display devices come in a variety of forms. A mobile telephone is a popular 
example, as is a personal digital assistant (PDA) or a handheld computer. This type of 
device generally includes a display screen for showing text or graphical information. 
With the recent introduction of the Wireless Application Protocol (WAP) and other 
types of software for mobile telephones and PDAs, such devices can be used to display 
Internet web pages. 

A disadvantage associated with a conventional handheld display device, is that, since its 
display screen is limited in physical size, the information which can be conveniently 
displayed is limited. For example, if a user wishes to view a document in the form of 
an image, it is generally only possible to view part of the image at any one time. In 
order to view a different part, the image has to be moved, for example by using an 
associated keyboard, which is cumbersome and repeatedly draws the user's attention 
away from the document. 

A known display device comprises a handheld body which includes a sensor surface 
situated beneath the body, the sensor surface being movable over a flat reference 
surface (e.g. a table). Control of the image part being displayed requires movement of 
the display device over the reference surface. This is clearly not desirable for the 
majority of handheld display devices, such as mobile telephones or PDAs. 

According to a first aspect of the present invention, there is provided a handheld display 
device comprising: a screen; a processor for outputting an image to the screen using 
image data from an image data source; and a sensor connected to the processor, the 
sensor being arranged to generate a varying sensor signal according to the free-space 
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position of the sensor, .the processor updating the image data being displayed in 
accordance with the varying sensor signal. 



Such a display device can update displayed image data according to the free-space 
5 position of the sensor. Accordingly, by orienting or moving the display device in a 
particular way, the image being displayed is updated to show some other set of image 
data. There is no requirement for a sensor surface, nor a reference surface for 
contacting the device. This is particularly convenient for devices such as mobile 
telephones or PDAs. 

10 

Preferably, the processor is arranged to output a sub-set of the image data thereby to 
display only part of an image to the screen, the image part being displayed changing in 
response to the varying sensor signal. In this sense, the image data being displayed 
relates to a sub-set of an overall image, e.g. a large document or web page. The varying 
1 5 sensor signal causes the processor to update the displayed image data. 

As an example, movement of the sensor in a particular direction may cause the 
processor to update the displayed image data by showing an image part partially 
adjacent to that previously displayed. Thus, movement of the display device has the 

20 effect of scrolling the displayed image part. Preferably, this is achieved by providing a 
tilt sensor which causes the displayed image part to scroll in accordance with the 
direction of tilt. The degree of tilt may also be utilised to determine the rate of update, 
which has the effect of increasing or decreasing the degree of perceived scrolling. 
Alternatively, this scrolling effect may be achieved by providing sensors, for example 

25 velocity sensors or accelerometers, that allow the horizontal position of the display 
device to be determined. Such a sensor would cause the displayed image part to scroll 
in proportion to the free-space position of the sensor, without requiring a sensor surface 
or reference surface over which the sensor moves. 



30 As a further example, movement of the sensor in first and second directions may cause 
the processor to update the displayed image data to show, respectively, a zoomed-in and 
zoomed-out version of the previously displayed image part. Thus, a user is able to 
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select a close-up view of a particular part of the image, or to zoom-out and select a 
different part of the image to zoom-in to. The first and second directions may 
correspond to upwards and downwards movements, respectively, along a generally 
vertical plane. 

According to a second aspect of the present invention, there is provided a hand held 
display device comprising: a screen; a processor for outputting an image part to the 
screen using image data from an image data source; and first and second sensors 
connected to the processor, the first and second sensors being arranged to generate, 
respectively, a first sensor signal which varies according to the free-space position of 
the hand held display device in a substantially vertical plane, and a second sensor signal 
which varies according to movement of the hand held display device in a further plane, 
the processor updating the image data being displayed in accordance with any one of 
the varying sensor signals. 

By providing such a display device having two sensors, the image part being displayed 
can be adjusted in one of two planes, or indeed, in both, at a given time. Preferably, the 
first sensor signal has the effect of zooming-in or zooming-out of the image part being 
displayed, whilst the second sensor signal has the effect of scrolling the image part 
being displayed to an adjacent region of the previously displayed image. . 

Various sensors may be employed, examples of which include infra-red or ultrasonic 
detectors, pickup coils and remote transmitters, mercury switches, velocity sensors and 
accelerometers. 

The invention will now be described, by way of example, with reference to the 
drawings, in which: 

Figure 1 is a block diagram showing the interrelationship of various elements of a 
handheld display device in the form of a PDA; 
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Figure 2 is an end-face view of the PDA, the PDA being shown in two different 
operating configurations; 

Figure 3 shows the screen of the PDA, an image being shown on the screen in normal 
5 form and in zoomed-in form; 

Figure 4 is an end-face view of the PDA, the PDA being shown in a tilted position; 

Figure 5 shows the screen of the PDA, with an image being shown at three different 
1 0 subsequent time periods; 

Figure 6 is a block diagram showing the interrelationship of various elements of a PDA 
having two sensors; 

1 5 Figure 7 shows the screen of the PDA, with an image being shown at two different 
levels of zoom and two different horizontal positions; 

Figure 8 is a block circuit diagram of an ultrasonic or infrared positioning sensing 
arrangement forming part of the PDA; 

20 

Figure 9 shows schematically the screen of the PDA and its relationship with image 
data stored in memory for two different positions of the PDA; and 

Figure 10 shows schematically the screen of the PDA with an image being shown at 
25 two different levels of zoom. 

Referring to Figure 1, a PDA 1 comprises a liquid-crystal display (LCD) screen 3, a 
processor 5, a sensor 7, a local memory unit 9, and a remote data input 11. The remote 
data input 1 1 comprises a conventional r.f. receiver. The processor 5 is arranged to 
30 receive remote data from an external image data source, by means of the remote data 
input 1 1 . The external image data source may, for example, be a remote data source 
which transmits image data over a wireless (e.g. cellular) link. The image data can be 
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stored in the memory unit 9 by the processor 5, and then output to the screen 3 by the 
processor 5; or, if the remote data link is fast enough, and the processor 5 controls the 
remote data source, the image data could be fed by the processor from the remote data 
input 1 1 to the screen 3. As will be explained in detail below, the displayed image data 
is updated in accordance with sensor signals received from the sensor 7. 

The processor 5 is provided with software for controlling various aspects of the signal 
processing which is required for updating the image in response to the received sensor 
signals. 

The local memory unit 9 can be used for temporarily storing data received from the 
external data source, or for storing other image information that is to be displayed on 
the display 3. Alternatively, remote data can be 'streamed' over a very fast data link 
between the external source and the remote input 1 1 . 

In use, the PDA 1 is used to display a variety of image documents which are accessible 
over a wireless link, or stored locally in the memory unit 9. These may comprise 
conventional Internet web-pages, maps, engineering drawings etc. Whatever their 
form, it will be appreciated that these forms of digitised image will generally be of a 
much higher resolution and/or size than that displayable by the screen 3. It follows that 
the PDA 1 will only display a small part of the image, at a given time, in order that the 
perceivable image is of a good quality. In order that a user may navigate around the 
entire image, i.e. by viewing only parts of the image in turn, the sensor 7 is arranged 
such as to update the image data being displayed, according to the free space position of 
the sensor. This may be done in a number of ways, as explained with reference to the 
following first, second and third embodiments. 

In the first embodiment, free-space movement of a sensor T (as shown in Figure 2) 
causes the screen 3 (under the control of the processor 5) to show zoomed-in and 
zoomed-out versions of the image part previously displayed. Thus, when a user wishes 
to navigate to. a particular part of an image, he/she may zoom-out to show the entire 
image (albeit in an substantially unreadable form) and then may zoom-in to the image 
part of interest. The central point of the image part to be zoomed-in at may be adjusted 
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by conventional means (such as navigation keys), or by using a further free-space 
sensor type, as will be explained later. 



Figures 2a and 2b show two different arrangements wherein the sensor is employed to 
5 cause the screen 3 to zoom-in and zoom-out over an image part. In both cases, 
movement of the PDA 1, and so the sensor 7', in a substantially vertical plane causes 
the zooming-in or zooming-out effect, depending on whether the movement is in an 
upwards or downwards direction. If the sensor T moves upwards, the processor 5 
causes a zooming-out effect on the screen 3, whilst downwards movement of the sensor 
1 0 results in a zooming-in effect. 

Figure 2a shows an end-face of the PDA 1 positioned above a reference surface 13, 
such as a desk or floor surface. The sensor T can be in the form of an ultrasonic or 
infrared detector which is located towards the rear of the PDA 1 . In this case, the 

15 sensor T determines its position in free space by measuring its distance from the 
reference surface 13.. If the distance increases over a particular time period, then it 
follows that the PDA 1 has moved upwards, and a suitably modulated sensor signal is 
output to the processor 5 for causing the displayed image to be updated so as to give a 
perceivable zooming-out effect. If the distance decreases, then the PDA 1 has moved 

20 downwards, and the sensor signal is modulated accordingly and output to the processor 
5, causing the updated image to give a perceivable zooming-in effect 

Referring to Figure 2b, a similar arrangement is shown. In this case, however, the 
sensor T is in the form of an accelerometer or a velocity sensor. It will be appreciated 

25 that such an accelerometer can determine whether the PDA 1 is moving upwards or 
downwards according to the instantaneous movement of the sensor. Indeed, any 
detector or sensor capable of detecting an instantaneous movement may be employed. 
If a velocity sensor is used, its output is integrated by the processor 5 to give a 
measurement of distance; and, if an accelerometer is used, its output is double 

30 integrated by the processor to give a measurement of distance. Again, if the sensor 7' 
determines that an upwards movement has occurred, a suitably modulated sensor signal 
is output to the processor 5 for causing the displayed image to be updated so as to give 



BNSDOCID: <GB 2378878A__I_> 



BEST AVAILABLE COPY 



7 

the zooming-out effect. If the sensor T is moved downwards, the sensor signal is 
modulated accordingly and output to the processor 5, causing the updated image to give 
the zooming-in effect. 

5 Referring to Figure 3, the zooming-in effect is shown. Figure 3a shows a simple scene 
displayed on the screen 3 of the PDA, whilst Figure 3b shows a zoomed-in version of 
the same scene. 

A second embodiment according to the present invention is shown in Figure 4, which 
1 0 shows an end-face of the PDA 1 . In this embodiment, free-space movement of a sensor 
7" causes the processor to show an adjacent part of the image previously displayed, i.e. 
to scroll the image in a particular direction. To facilitate this, the sensor 7" comprises a 
tilt sensor for detecting when the PDA 1 is tilted, and in which direction. The sensor 
7" is capable of determining a tilting effect in four directions, i.e. left, right, forwards, 
15 and backwards, although for ease of explanation, only the effect of a left and right tilt is 
mentioned below. 

When the PDA 1, and so the sensor T\ is tilted to the left (as shown in Figure 4) the 
processor 5 updates the screen 3 to show an adjacent region to the left-hand side of the 

20 previously displayed image. This process is repeated at a particular update rate until the 
PDA 1 is tilted to be horizontal. It follows that a tilt movement to the right will result 
in the image being updated to show an adjacent region to the right-hand side of the 
previously-displayed image. In use, the sensor arrangement enables a user to navigate 
over different parts of a large image, simply by orienting (in this case, by tilting) the 

25 PDA 1 in free-space. 

Figure 5 illustrates the abovementioned updating effect caused by the tilting motion 
shown in Figure 4. The updated image is shown at three consecutive time periods, tj, t 2 , 
and t 3 . It will be seen that the updated image at t 2 comprises a large quantity of image 
30 data from the previous image at t l? the amount of actual 'shift' being small. This shift 
amount is determined by the software in the processor 5, and helps provide a smooth 
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scrolling effect rather than a 'jerky' movement of the image. The updated image at t 3 
shows further movement in the same direction. 



In relation to both the first and second embodiments, the software in the processor 5 
5 handles various signal processing operations for controlling the updating of image data. 
These signal processing operations not only control the respective zooming and 
scrolling effects, but also the rate at which the updating occurs. Indeed, in order for the 
user to have the maximum control over the image navigation process, it is preferable 
that the degree of movement of the PDA 1 be associated with the rate of image data 
1 0 update. Accordingly, in the case of the first embodiment, the speed of movement of 
the sensor T in the vertical plane determines the speed of the perceivable zooming 
effect. The faster the speed of movement, the faster the zooming effect. In the case of 
the second embodiment, the degree of tilt of the PDA 1 determines the speed of the 
perceivable scrolling. The higher the degree of incline, the faster the scrolling effect. 

15 

A third embodiment according to the present invention is shown in Figure 6. This 
embodiment effectively combines the operation of the first and second embodiments, 
and so like elements between Figures 1 and 6 are referred to by the same reference 
numerals. In this embodiment, a PDA 15 is provided with first and second sensors 17, 
20 1 9, the first sensor being an accelerometer, and the second sensor being a tilt sensor. 

The operation of the processor 5 is similar to that described in relation to the first and 
second embodiments, in that the image is updated according to a sensor signal. 
However, in this case, two sensor signals are provided from the first and second sensors 

25 17, 19. The sensors 17, 19 are positioned and function in the same way as described 
previously. The processor 5 updates the image displayed in accordance with these two 
sensor signals, i.e. so that movement of the first sensor 17 causes a zooming effect on 
the image, whilst tilting of the second sensor 19 causes a scrolling effect on the image. 
Hence, the improved image navigation provided for by the first and second 

30 embodiments is improved further by using both navigation methods. 
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Accordingly, the overall effect of using both sensors 17, 19, is that the user can 
'fly-over' the entire image by means of both moving the PDA 15 in a vertical plane, and 
also by tilting the PDA. Hence, the simple, intuitive act of raising the PDA 15, tilting 
it, and then lowering it results in the image being zoomed-out, scrolled to a selected 
5 part, and then zoomed-in to show a detailed view of that selected image part. In effect, 
the PDA 15 acts in a manner similar to that of a magnifying glass tilted and moved over 
a virtual document. Again, the speed of movement and the degree of tilt will determine 
the image update rate to control the speed of zoom and/or the scrolling speed, the 
appropriate signal processing being performed by software in the processor 5. This 
10 software should provide for a maximum degree of realism, i.e. by closely matching the 
real-time scrolling and zooming of the image with the actual movement of the PDA 15. 
Ultimately, the user should be convinced that he is 'gliding' in real-time above the 
surface of a virtual document. 

15 As an alternative to the tilt sensor 7" and 19, position sensors, velocity sensors or 
accelerometers may be used. These would be mounted at 90° to each other, both in the 
nominally horizontal plane, such that the first (for example) detected position, velocity 
or acceleration is the left-to-right direction of the PDA, and the second detected 
position, velocity or acceleration is the forwards-and-backwards direction. The image 

20 on the display would be scrolled in accordance with the lateral position of the display as 
detected by the sensors. If the display is "zoomed in" by moving it downwards (see 
Figure 2b) or towards a surface (see Figure 2a), then a lateral movement of 10 mm in 
the position of the display would result in a similar movement of the image on the 
display, i.e. 10 mm. If the display is "zoomed out" by moving it upwards (see Figure 

25 2b) or away from a surface (see Figure 2a), then a lateral movement of 10 mm in the 
position of the display would result in a larger movement of any specific feature of the 
image on the display. However, as that image is shown on the display in "zoomed out" 
form, i.e. at a lower resolution, the actual image seen would again appear to move in 
proportion of the movement to the display in free space. 

30 

For example, as shown in Figure 7, a horizontal movement of the display of 10 mm to 
the left when "zoomed in" may result in the image shown in Figure 7a changing to the 
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image shown in Figure 7b. However, when "zoomed out", as shown in Figure 7c, the 
same movement of the display, i.e. 10 nun to the left, may then show the image shown 
in Figure 7d. In Figure 7d, the image has moved, compared to that shown in Figure 7c, 
by a similar amount but, being "zoomed out", the user can see much more "new" 
5 information on the display than when moving from Figure 7a to Figure 7b. 

Figure 8 illustrates one way in which the PDA can manage the signal processing 
required to translate sensor signals into a zooming and/or scrolling effect effect. Thus, 
the signal processing required to produce a zooming and/or scrolling effect consists of 
10 the processing of the sensor signal in order to derive the sensor's position, and the use 
of that signal to affect the displayed image. 

As mentioned above, different types of sensor can be used for this application, 
principally those that measure position, velocity, acceleration, or tilt (i.e. orientation). 

15 

The use of non-contacting position or distance-measuring sensors is well established in 
electronics - for example, in infrared or ultrasonic rangefinders for cameras, or in 
ultrasonic 'tape measures' for measuring the size of rooms. These sensors use a variety 
of measurement techniques - optical, ultrasonic, inductive or capacitive coupling, etc. 

20 

In the arrangement shown in Figure 8, a positioning-sensing technique () will be 
described. This will be described as using an ultrasonic signal. However, it will be 
appreciated that the use of an infrared signal would use a very similar technique. A 
pulse generator 21 produces frequent repetitive pulses, e.g. 1000 times per second. 
25 These are fed to an ultrasonic transmitter 22, which transmits a short ultrasonic pulse 
towards a reference surface 23. The pulse generator 21 also resets a timing circuit 24, 
and starts a new timing measurement t T . The timing circuit 24 proceeds to count the 
number of pulses received at a clock input 25 from a high speed clock 26 whose 
frequency is f c . 

30 

The transmitted ultrasonic signal is reflected off the reference surface 23, and detected 
by a receiver 27. This amplifies the received pulse t R , and feeds it to the timing circuit 
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24. The pulse stops the counting of the high speed clock 26, and is fed to the processor 
5 (not shown in Figure 8) to tell it that a new measurement is ready. 

The timing circuit 24, therefore, measures the time t D between the transmit pulse and the 
receive pulse (t D = t R -tT) with a resolution of l/f c , and the output of the timing circuit is 
a counter value Co which equals the high speed clock frequency f c multiplied by the 
time interval t D . The counter value C 0 is read by the processor 5 after receiving the 
received pulse t R . 

The count value Co is proportional to the distance D between the transmitter/receiver 
22/27 and the reference surface 23. As the transmitter/receiver 22/27 move closer to 
the reference surface 23, the value of C 0 decreases; and, as the transmitter/receiver 
move away from the reference surface the value of Co increases. C 0 is, therefore, a 
measure of distance of the sensors from a given reference surface. 

Clearly, if two such sensors were mounted orthogonally, they would enable the position 
in a plane - with respect to reference surfaces - to be derived; and, if three were 
mounted and used orthogonally, they would enable the position in three-dimensional 
space to be derived. 

It would also be possible to use other configurations, e.g. a fixed transmitter in place of 
the reference surface 23, and simply measure the distance from this to the receiver 27 
in a similar way to that indicated, although it would have to feed a start pulse f T to the 
timing circuit 24 by an alternative means that was not significantly distance dependent. 
However, additional signal processing would then be required. 

It will be apparent that the position sensor described above could be modified in a 
number of ways. In particular, the transmitter/receiver pair 22/27 could operate with 
infared pulses rather than ultrasonic pulses. It would also be possible to use a velocity 
sensor or an accelerometer in place of the position sensor described. The signal 
processing requirements of velocity and accelerometer sensors are very similar. Both 
types of sensor require amplification before integration. 
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In the case of a velocity sensor, a single stage of integration is required to derive 
distance; and, for an accelerometer, two stages of integration (i.e. double integration) 
are required to derive distance. Although the integration can be performed using 
analogue electronics, it is preferable to use the processor 5 to perform the integration, as 
this is most easily, reproducibly, and flexibly done via a software program. 

The result of the single (or double) integration will be a value that is proportional to the 
position of the sensor in that particular axis with respect to its starting point. Clearly, if 
two such sensors are mounted orthogonally, they will enable the position in a plane to 
be derived; and, if three are mounted orthogonally, they will enable the position in 
three dimensional space to be derived. 

When using the tilt sensor T of Figure 4, this can be in proportional form, e.g. a 
potentiometer or encoder with a pendulum attached; or in switched form, e.g. a 
mercury switch or a ball bearing running over curved switch contacts. A single tilt 
sensor can determine the orientation of a device in one axis. In the case of a 
proportional sensor, this will give an output proportional to the amount by which it is 
tilted, e.g. from the horizontal. A switched sensor usually indicates one of a limited 
number of states - e.g. tilted left/horizontal/tilted right (three-position switch), or tilted 
very left/tilted slightly lefi/horizontal/tilted slightly right/tilted very right (five-position 
switch). 

Clearly, if two tilt sensors are mounted orthogonally, they will enable the orientation in 
two orthogonal axes, i.e. in a plane, to be determined. 

The orientation of the tilt sensors can be used to derive the apparent position of the 
device as follows (note, only one axis will be described but the same principle would be 
applied to both axes, i.e. to the two orthogonally-mounted sensors). The output of the 
tilt sensor 7" is fed to the processor 5. The processor 5 takes no action if the sensor 
indicates horizontal. If the sensor T indicates that it is tilted slightly to the left, the 
processor 5 starts a counter counting slowly in a negative direction. The greater the 
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angle of tilt, the faster the negative counting. Clearly, for a simple three-position tilt 
switch there is only one fixed rate of counting in each direction; whereas, for a 
five-position tilt switch, there can be two different rates of counting in each direction. 
If the sensor 7" is returned to horizontal, the counting stops. 

If the sensor 7" indicates that it is tilted slightly to the right, the processor 5 starts the 
counter counting slowly in a positive direction. The greater the angle of tilt, the faster 
the positive counting. 

The value of the counter is proportional to the apparent position of the device in that 
axis, and it is this counter output that is used to feed to the processor 5 and affects the 
displayed image as described below. 

Hence, tilting the sensor 7" left by 30° for two seconds, returning it to the horizontal, 
then tilting to the right by 30° for two seconds, and then returning it to the horizontal 
would result in the device's apparent position starting in the centre, then moving to the 
left at a fixed rate for two seconds, then stopping, then moving to the right at a fixed 
rate for two seconds back to its starting position, and then stopping again. 

More sophistication can be added by altering the counting rate, depending not only on 
angle of tilt but also on duration of tilt (time), and in a non-linear way (e.g. by altering 
the apparent nature of the horizontal 'dead band' in which the moving stops), to make it 
easier for the user to move the apparent position of the device wherever chosen. 

To describe the manner in which signal processing is used to affect the displayed 
image, let us assume that the image is a large star shape, as shown in Figure 9a, and let 
us assume that the image of the star is too large to all be shown in the display (the 
screen 3) at maximum resolution. 

Data corresponding to the complete image of the star will be held either in the memory 
unit 9, or in a remote data source connected via the remote data input 11. The 
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processor 5 selects the portion of the image to be displayed at any one time, and feeds 
the appropriate image data to the display 3. 

The exact method by which an image is represented in memory can vary; but, for this 
5 example, let us assume the most common method, i.e. a direct 'bit-mapping' between 
the memory location and the image, as shown in Figure 9b, with each memory location 
corresponding to one pixel on the display 3. The memory location could be locally in 
the memory unit 9, or accessed remotely via the remote data input 1 1 . (Note that the 
approach proposed in this example is equally applicable, with suitable amendments of 
10 the specific details of operation, to many other ways of representing an image within a 
data storage device.) 

In Figure 9b, the memory is 64 kbytes, and is represented as being arranged in 256 
'lines' of 256 memory addresses per line, i.e. a 256 x 256 array. The display 3 is 128 
15 pixels x 128 pixels. The processor 5 can take the contents of the memory at various 
addresses, and feeds these to the appropriate pixel of the display 3. By choosing 
memory addresses whose positions in the 256 x 256 array correspond to the required 
positions on the display 3, a direct mapping of part or all of the image stored in memory 
to the image shown on the display can be achieved. 

20 

For the image shown in the display 3 of Figure 9a, the processor 5 would send the data 
from memory locations 16,191 to (16,191+128) to the first line of pixels on the display, 
memory locations (16,191+256) to (16,191+256+128) to the second line of pixels on 
the display, and so on, through to the final pixel on the final line of the display, 49,087, 
25 as shown in Figure 9b. 

If the sensors, after suitable signal processing as described above, indicate that they had 
been moved by, say, one quarter of the width (equivalent to 32 memory locations) and 
one quarter of the height (equivalent to 32 memory locations) of the display 3 to the 
30 bottom right, we would expect the image shown on the display to be as shown in Figure 
9c. 



BNSDOCID: <GB 2378878A_I_> 



15 

For the image shown in the display 3 of Figure 9c, the processor 5 would send the data 
from memory locations 24,415 to (24,415+128) to the first line of pixels on the display, 
memory locations (24,415+256) to (24,415+256+128) to the second line of pizxels on 
the display, and so on, through to the final pixel on the final line of the display, 57,31 1, 
5 as shown in Figure 9d. 

It can, therefore, be seen that, in each axis, the portion of the memory to be displayed 
has a start point and a subsequent pattern of memory locations that are used to drive the 
display with pixel data, and it is this start point that is varied according to the processed 
10 sensor information. The greater the value from the sensors, i.e. the greater the 
movement of the sensors in that direction, the greater the off-set between the 'normal' 
(initial state of the display) start point, e.g. 16,191 in Figure 9b, and the new start point 
24,415 as shown in Figure 9d. 

15 It should be understood that the image would not usually move from the position shown 
in Figure 9a to the position shown in Figure 9c instantaneously in one step, as the 
sensors would not move so quickly. The processor 5 will monitor the sensors 
frequently (e.g. thousands of times per second); and, as the value from the sensors 
change, it will vary the memory locations it sends to the display 3 at the same rate in 

20 proportion to the sensor variations, giving the effect of the image moving in the display 
in relationship to the display and sensor movement in space. 

So, even if the sensors are moved quickly, the high repetition rate at which the sensors 
are monitored and the display 3 updated means that the difference in position of the 
25 image on each update of the display would only be small. The result would, therefore, 
appear as a smooth scrolling action, i.e. the movement of the image from the position 
shown in Figure 9a to that shown in Figure 9c would take place smoothly in a number 
of stages at a substantially similar rate to the speed of movement of the sensors. 

30 By matching the movement of the image in the display 3 very accurately to the exact 
physical movement of the display and sensor in space, the impression will be given that 
the image is fixed in space, and the display is merely passing over it, and allowing the 
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viewer to see a portion of the fixed image through a 'moving window'. It is 
important that the apparent movement of the image matches the action taken by the user 
of the. device very closely, as the illusion of viewing a fixed document through a 
moving window is easily destroyed if the matching is not accurate. 

Figures 9 and 10 illustrate how the PDA manages the signal processing required to 
translate sensor signals into a zooming effect. Thus, the signal processing required to 
produce a zooming effect consists of processing of the sensor signal in order to derive 
the sensor's height (vertical position), and the use of that signal to affect the displayed 
image. 

Different types of sensor can be used for this application, principally those that measure 
position (e.g. distance from a fixed surface), velocity, or acceleration. 

The signal processing requirements of the position, velocity and accelerometer sensors 
are as described above for the scrolling effect. In order to describe the manner in which 
signal processing is used to affect the displayed image, let us assume that the image is a 
large star, as shown in Figure 9a, and let us assume that the star is too large to be 
completely shown in the display 3. The complete image of the star will be held either 
in the memory unit 9, or in a remote data source connected via the remote data input 1 1 . 
The processor 5 selects the portion of the image to be displayed at any one time, and 
feeds the appropriate image data to the display 3. 

The exact method by which an image is represented in memory can vary; but, for this 
example, let us assume the most common method, i.e. a direct mapping between 
memory location and image, with one memory location representing one pixel on the 
display 3, as shown in Figure 9b. For the nominally-sized image shown in the display 3 
of Figure 9a, the processor 5 would send the data from memory locations 16,191 to 
(16,191+128) to the first line of pixels on the display, memory locations (16,191+256) 
to (16,191+256+128) to the second line of pixels on the display, and so on, through to 
the final memory address on the final line, 49,087. 
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If the sensors, after suitable signal processing as described above, indicate that they had 
been moved upwards by, say 50 mm, we would expect the image to zoom out, and be 
shown on the display 3 as indicated in Figure 10a. For the image shown in the display 
of Figure 10a, the processor 5 would have to utilise a wider range of data from the very 
first memory location 0 to the final memory address on the final line, 65,535, to the 
display as shown in Figure 10b; and, since the display could not display all this data 
(there are 65,536 memory locations but only 16,384 pixels in the display) the processor 
5 would only send alternate x-axis memory locations to the display, and would omit 
alternate rows to compress the image in the y-axis, so that only one in four memory 
locations is shown on the display. 

If the sensors were moved even further upwards, then an even smaller image would be 
displayed by having the processor 5 show even fewer memory locations, e.g. sending 
only one out of every four x-axis memory locations to the display 3, and only one in 
every four rows, to compress the image so that only one in sixteen memory locations is 
shown on the display. 

Enlargement of the image, i.e. zooming in, as shown in Figure 10c can be achieved in 
two ways, namely:- 

1. By having the 'nominal-sized' image, e.g. as shown in Figure 9a, at lower 
resolution than the image data stored in memory (or remotely), so that the processor 5 is 
already showing only one in four memory locations, for example. In this instance, 
'zooming in' would actually mean showing the image at its full resolution. 

2. If the 'nominal-sized' image, e.g. as shown in Figure 9a, is already at 
maximum resolution, i.e. one pixel is represented by one memory location, then various 
well-known interpolation algorithms may be applied by the processor 5 to increase the 
apparent size of the image. For example, the simplest is for each x axis memory 
location to be fed to two adjacent pixels, and for each y axis line to be repeated. This 
increases the y axis image size (albeit with no increase in actual image resolution). 
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The processor 5 will, therefore, monitor the sensor frequently (e.g. hundreds of times 
per second); and, as the signal processed value from the sensor changes, it will vary not 
just the start and end points of the memory locations it sends to the display 3, but also 
how many of the memory locations between the start and end points it sends to the 
display, in proportion to the sensor variations, giving the effect of the image zooming in 
and out in relationship to the display and sensor movement in space. 

This zooming effect can, of course, be combined with the scrolling effect. 
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Claims 

1 . A handheld display device comprising: a screen; a processor for outputting an 
image to the screen using image data from an image data source; and a sensor 
connected to the processor, the sensor being arranged to generate a varying sensor 
signal according to the free-space position of the sensor, the processor updating the 
image data being displayed in accordance with the varying sensor signal. 

2. A handheld display device according to claim 1, wherein the processor is 
arranged to output a sub-set of the image data thereby to display only part of an image 
to the screen, the image part being displayed changing in response to the varying sensor 
signal. 

3. A handheld display device according to claim 2, wherein movement of the 
sensor in first and second directions causes the processor to update the displayed image 
data to show, respectively, a zoomed-in and a zoomed-out version of the previously 
displayed image part. 

4. A handheld display device according to claim 3, wherein the sensor is arranged 
such that variation of the sensor signal depends on the distance of the sensor from a 
reference surface. 

5. A handheld display device according to claim 3, wherein the sensor is arranged 
such that variation of the sensor signal depends on the instantaneous movement of the 
sensor. 

6. A handheld display device according to claim 2, wherein the sensor is arranged 
such that movement of the sensor in a particular direction causes the processor to 
update the displayed image data by showing an image part overlapping that previously 
displayed. 



5 
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7. A handheld display device according to claim 6, wherein the sensor is a tilt 
sensor. 

8. A handheld display device according to claim 7, wherein the rate at which the 
processor updates the displayed image depends on the degree of tilt of the tilt sensor. 

9. A handheld display device according to claim 6, wherein the sensor is a velocity 
sensor, a position sensor or an accelerometer. 

10 10. A handheld display device according to claim 9, wherein the processor is such 
that the rate at which it updates the displayed image depends upon the rate of change of 
the position of the device as indicated by the sensor. 



15 



11. A handheld display device comprising: 
a screen; 

a processor for outputting an image part to the screen using image 
data from an image data source; and 
first and second sensors connected to the processor, 
the first and second sensors being arranged to generate, respectively, a first sensor 
20 signal which varies according to the free-space position of the handheld display device 
in a substantially vertical plane, and a second sensor signal which varies according to 
movement of the handheld display device in a further plane, the processor updating the 
image data being displayed in accordance with any one of the varying sensor signals. 

25 12. A handheld display device according to claim 11, wherein the. sensor is 
arranged such that variation of the first sensor signal causes the processor to update the 
image data being displayed by means of displaying a zoomed-in or a zoomed-out 
version of the previously displayed image part, depending on the direction of movement 
in the substantially vertical plane. 



30 



13. A handheld display device according to claim 11 or claim 12, wherein the 
sensor is arranged such that variation of the second sensor signal causes the processor 
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to update the image data being displayed by means of displaying an image part which is 
adjacent to the previously displayed image part. 

1 4. A handheld display device according to any preceding claim, wherein the image 
5 data source is an external data source. 

15. A handheld display device according to any preceding claim, wherein the device 
is in the form of a portable telephone. 

10 16. A handheld display device according to any of claims 1 to 15, wherein the 
device is in the form of a PDA. 

17. A handheld display device, substantially as hereinbefore shown and described 
with reference to the accompanying drawings. 
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